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(54) Image pickup apparatus 

(57) An image pickup apparatus comprising a plu- 
rality of openings for taking in external light at different 
positions, filter for extracting from external light taken in 
from the plurality of openings, predetermined wave- 
length components of subject light correspondingly to 
the plurality of openings, respectively, and a plurality of 
image pickup portions for receiving the predetermined 
wavelength components extracted by the filter corre- 
spondingly to the plurality of openings, respectively. 
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Description 

BACKGROUND OF THE INVENTION 
Field of the Invention 

[0001] The present invention relates to an image pick- 
up apparatus such as a video camera, capable of pick- 
ing up a dynamic image or a still image. 

Related Background Art 

[0002] In a digital color camera, in response to de- 
pressing of a RELEASE button, an image in a field is 
exposed to a solid-state image pickup element such as 
a CCD or a CMOS sensor for a desired time period, im- 
age signal obtained by this, representing a still image of 
one picture is converted into digital signal and is subject 
to a predetermined processing such as the YC process- 
ing, and image signal in a predetermined format are ob- 
tained. Digital image signal representing the picked-up 
image is recorded in a semiconductor memory on a pic- 
ture basis. The recorded image signal is read whenever 
necessary to be reproduced as a signal capable of being 
displayed or printed, and is outputted to a monitor or the 
like to be displayed. 

[0003] As a technology for a digital color camera, one 
disclosed in Japanese Patent Application Laid-open No. 
7-123418 is known. Japanese Patent Application Laid- 
open No. 7-123418 discloses that an object image on 
an estimated imaging plane is reimaged by a plurality of 
reimaging optical systems, with the respective object 
images being picked up through optical filters having dif- 
ferent spectral transmission factor characteristics. This 
is suitable for obtaining a color image with high resolu- 
tion using image pickup elements of a limited number of 
pixels. 

[0004] However, the above structure has a disadvan- 
tage that color misalignment of an object image is 
caused which depends on the imaging state of the ob- 
ject. This is caused because the light beams of the plu- 
rality of reimaging optical systems are separated on the 
pupil of an objective lens. It also has a disadvantage that 
the camera has to become larger in order to reimage an 
image of an object. 

SUMMARY OF THE INVENTION 

[0005] According to one aspect of the present inven- 
tion, there is provided an image pickup apparatus com- 
prising a plurality of openings for taking in external light 
at different positions, a filter for extracting from external 
light taken in from the plurality of openings, predeter- 
mined wavelength components of object light corre- 
spondingly to the plurality of openings, respectively, and 
a plurality of image pickup portions for receiving the pre- 
determined wavelength components extracted by the fil- 
ter correspondingly to the plurality of openings. The im- 



age pickup apparatus of the present invention is small- 
sized, has a short image pickup optical path, and yet 
can obtain a desired color image. 
[0006] Other aspects of the present invention will be- 
5 come apparent from the following preferable specific 
embodiments. 

BRIEF DESCRIPTION OF THE DRAWINGS 
10 [0007] 

Fig. 1 is a sectional view of an image pickup system 
of a digital color camera according to an embodi- 
ment of the present invention; 
15 Fig. 2 is a front view of a solid-state image pickup 
element of the image pickup system shown in Fig. 1 ; 
Fig. 3 is a plan view of a diaphragm of the image 
pickup system shown in Fig. 1; 
Fig. 4 illustrates the ranges where optical filters of 
the image pickup system shown in Fig. 1 are 
formed; 

Fig. 5 illustrates an objective lens of the image pick- 
up system shown in Fig. 1 which is viewed from the 
side where light is projected; 
Fig. 6 is a graph showing the spectral transmission 
factor characteristics of the optical filters of the im- 
age pickup system shown in Fig. 1; 
Fig. 7 is a graph showing the spectral transmission 
factor characteristics of a color purity correction fil- 
ter of the image pickup system shown in Fig. 1 ; 
Fig. 8 is a graph showing the spectral transmission 
factor characteristics of a photochromic glass of the 
image pickup system shown in Fig. 1; 
Fig. 9 is a graph showing the transmission factor 
distribution of a transmission factor distribution type 
filter of the image pickup system shown in Fig. 1; 
Fig. 10 is a graph showing the OTF characteristics 
of the objective lens of the image pickup system 
shown in Fig. 1; 

Fig. 11 is a graph showing the intensity distribution 
in an image by the image pickup system shown in 
Fig. 1; 

Fig. 1 2 is an explanatory view for explaining the set- 
ting of the intervals between lens portions of the im- 
age pickup system shown inFig. 1 ; 
Fig. 1 3 is an explanatory view for explaining the po- 
sitions of images of an object at infinity of the image 
pickup system shown in Fig. 1; 
Fig. 14 is a block diagram of a signal processing 
system of the digital color camera shown in Fig. 1 ; 
Fig. 15 illustrates the positional relationship of the 
image pickup region for R and B pixels with respect 
to the image pickup region for a G pixel of the image 
pickup system shown in Fig. 1; 
Fig. 16 is an explanatory view of interpolation 
processing of the digital color camera shown in Fig. 
1; 

Figs. 17A, 17B, and 17C illustrate the overall struc- 



25 



30 



35 



40 



45 



50 



3 



EP 1 067 779 A2 



4 



ture of the digital color camera shown in Fig. 1 ; 
Fig. 1 8 is a detailed view of an image pickup system 
of a digital color camera according to a second em- 
bodiment of the present invention; 
Fig. 19 illustrates an objective lens of the image 5 
pickup system shown in Fig. 18 which is viewed 
from the side where light enters; 
Fig. 20 illustrates the objective lens of the image 
pickup system shown in Fig. 19 which is viewed 
from the side where light is projected; 10 
Fig. 21 is a plan view of a diaphragm of the image 
pickup system shown in Fig. 18; 
Fig. 22 is a front view of a solid-state image pickup 
element 820 of the image pickup system shown in 
Fig. 18; 15 
Fig. 23 is an explanatory view for explaining the set- 
ting of the intervals between lens portions of the im- 
age pickup system shown in Fig. 18; and 
Fig. 24 is an explanatory view for explaining the po- 
sition of an image of an object located at infinity of 20 
the image pickup system shown in Fig. 18. 

DESCRIPTION OF THE PREFERRED 
EMBODIMENTS 

25 

[0008] Preferred embodiments of the present inven- 
tion are now described in detail with reference to the 
drawings. 

[0009] Figs. 17A, 17B, and 17C illustrate the overall 
structure of a digital color camera according to a first 30 
embodiment of the present invention. Figs. 17A, 17C, 
and 17B are a front view, a rear view, and a sectional 
view taken along the line 17B-17B in the rear view of 
Fig. 1 7C, respectively. 

[0010] In Figs. 17A, 17B, and 17C, reference numeral 35 
1 denotes a camera body. An illuminating light intake 
window 2 formed of a white diffusing plate is positioned 
at the back of a color liquid crystal monitor 4. Reference 
numerals 5 and 6 denote a main switch and a RELEASE 
button, respectively. Reference numerals 7, 8, and 9 are *o 
switches for a user to set the state of the camera. In 
particular, the reference numeral 9 denotes a PLAY but- 
ton. A reference numeral 13 denotes a display of the 
remaining number of recordable pictures. Object light 
which is incident into a prism 12 from a viewfinder front 45 
frame 3 is projected from a viewfinder eyepiece window 
11. Reference numerals 10 and 14 denote an image 
pickup system and a connection terminal for connecting 
to an external computer or the like to transmit and re- 
ceive data, respectively. so 
[0011] The schematic structure of a signal processing 
system is now described. Fig. 14 is a block diagram of 
the signal processing system. This camera is a single 
plate type digital color camera using a solid-state image 
pickup element 1 20 such as a CCD or a CMOS sensor. 55 
Image signals representing a dynamic image or a still 
image are obtained by driving the solid-state image pick- 
up element 120 continuously or only once. Here, the sol- 



id-state image pickup element 120 is an image pickup 
device of the type which converts exposed light into 
electric signals with regard to the respective pixels, 
stores respective charges according to the light amount, 
and reads the respective charges. 
[0012] It is to be noted that Fig. 14 only shows portions 
which are directly relevant to the present invention, and 
illustration and description of portions which are not di- 
rectly relevant to the present invention are omitted. 
[0013] As shown in Fig. 14, the image pickup appara- 
tus has the image pickup system 1 0, an image process- 
ing system 20 as an image processing means, a record- 
ing and reproduction system 30, and a control system 
40. Further, the image pickup system 10 includes there- 
in an objective lens 1 00, a diaphragm 110, and the solid- 
state image pickup element 120, the image processing 
system 20 includes therein an A/D converter 500, an 
RGB image processing circuit 21 0, and a YC processing 
circuit 230, the recording and reproduction system 30 
includes therein a recording processing circuit 300 and 
a reproduction processing circuit 310, and the control 
system 40 includes therein a system control unit 400, 
an operation detection circuit 410, and a drive circuit 420 
of the solid-state image pickup element. 
[0014] The image pickup system 10 is an optical- 
processing system for imaging light incident from an ob- 
ject through the diaphragm 110 and the objective lens 
100, on an image pickup plane of the solid-state image 
pickup element. The light transmission factor of the ob- 
jective lens 100 is adjusted to expose a subject image 
of an appropriate light amount to the solid-state image 
pickup element 120. As described in the above, as the 
solid-state image pickup element 120, an image pickup 
device such as a CCD and a CMOS sensor is applied. 
By controlling the exposure time and the exposure in- 
tervals of the solid-state image pickup element 120, im- 
age signals representing continuous dynamic images or • 
image signals representing a still image by one expo-" 
sure can be obtained. 

[0015] Fig. 1 is a detailed view of the image pickup 
system 10. First, the diaphragm 110 has three circular 
openings 110a, 110b, and 110c as shown in Fig. 3. Ob- 
ject light which enters a light incidence plane 100e of 
the objective lens 100 from the openings are projected 
from three lens portions 100a, 100b, and 100c of the 
objective lens 100 to form three object images on the 
image pickup plane of the solid-state image pickup ele- 
ment 1 20. The diaphragm 110, the light incidence plane 
100e, and the image pickup plane of the solid-state im- 
age pickup element 1 20 are in parallel with one another. 
In this way, by making weak the power on the incidence 
side, making strong the power on the projection side, 
and providing a diaphragm on the incidence side, the 
curvature of the image plane can be made less. It is to 
be noted that, though the light incidence plane 100e of 
the objective lens 100 is a plane here, the light incidence 
plane 100e may be formed of three spherical surfaces 
or three rotation-symmetry nonspherical surfaces. 
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[0016] The three lens portions 100a, 100b, and 100c 
have circular spherical surface portions as shown in Fig. 
5 showing the objective iens 100 viewed from the side 
where light is projected. An infrared radiation cutting fil- 
ter having a low transmission factor with regard to the 
wavelength range of 670 nm and more is formed on the 
spherical surface portions while a light-shielding layer 
is formed on a plane portion 100d shown with hatching. 
More specifically, the objective optical system is formed 
with the objective lens 1 00 and the diaphragm 1 1 0, while 
the three lens portions 100a, 100b, and 100c are the 
imaging system. 

[0017] The manufacturing is easy with using glass 
molding in case the objective lens 100 is made of glass 
and with using injection molding in case the objective 
lens 100 is made of resin. 

[0018] Fig. 2 is a front view of the solid-state image 
pickup element 120. The solid-state image pickup ele- 
ment 120 is provided with three image pickup regions 
120a, 120b, and 120c corresponding to the formed three 
object images. Each of the image pickup regions 120a, 
12b, and 120c are sized to be 2.24 mm x 1 .68 mm with 
the arrangement of 800 x 600 pixels having the vertical 
and horizontal pitch of 2.8 pm. The size of the whole 
image pickup regions is 2.24 mm x 5.04 mm. The diag- 
onal size of each of the image pickup regions is 2.80 
mm. In the figure, image circles 51a, 51b, and 51c within 
which an object image is formed are circular, the sizes 
of which circles are determined by the size of the open- 
ings of the diaphragm and the spherical surface portions 
on the projection side of the objective lens 100. The im- 
age circles 51a and 51b overlap each other and the im- 
age circles 51 b and 51c overlap each other. 
[0019] In Fig. 1, hatched portions 52a, 52b, and 52c 
sandwiched between the diaphragm 110 and the objec- 
tive lens 100 are optical filters formed on the light inci- 
dence plane 100e of the objective lens 100. As shown 
in Fig. 4 showing the objective lens 100 viewed from the 
light incidence side, the optical filters 52a, 52b, and 52c 
are formed in regions which completely include the dia- 
phragm openings 110a, 110b, and 110c, respectively. 
[0020] The optical filter 52a has spectral transmission 
factor characteristics shown as G in Fig. 6 which mainly 
transmit green, the optical filter 52b has spectral trans- 
mission factor characteristics shown as R which mainly 
transmit red, and the optical filter 52c has spectral trans- 
mission factor characteristics shown as B which mainly 
transmit blue, and thus, these are primary color filters. 
As the product with the characteristics of the infrared 
radiation cutting filter formed on the lens portions 100a, 
100b, and 100c, the object images formed in the image 
circles 51 a, 51 b, and 51 c are of green light component, 
red light component, and blue light component, respec- 
tively. 

[0021] On the other hand, optical filters 53a, 53b, and 
53c are formed on the three image pickup regions 120a, 
120b, and 120c, respectively, of the solid-state image 
pickup element 120. Their spectral transmission factor 



characteristics are equivalent to the ones shown in Fig. 
6. More specifically, the image pickup regions 120a, 
120b, and 1 20c are sensitive to green light (G), red light 
(R), and blue light (B), respectively. 
5 [0022] Since the spectral distributions of the received 
light of the respective image pickup regions are given 
as the product of the spectral transmission factors of the 
pupils and the spectral transmission factors of the image 
pickup regions, respectively, the combination of the pu- 
10 pji and the image pickup region is selected according to 
the wavelength range. More specifically, object light 
which goes through the diaphragm opening 110a is 
mainly photoelectrically converted in the image pickup 
region 120a, object light which goes through the dia- 
ls phragm opening 110b is mainly photoelectrically con- 
verted in the image pickup region 120b, and object light 
which goes through the diaphragm opening 110c is 
mainly photoelectrically converted in the image pickup 
region 120c. In other words, the image pickup regions 
120a, 120b, and 120c output a G image, an R image, 
and a B image, respectively. In this way, by using mul- 
tiple optical filters for the purpose of color separation on 
the pupils of the image pickup optical system and on the 
image pickup element, the color purity can be en- 
hanced. This is because, by using the same kind of op- 
tical filters twice, the buildup of the transmission factor 
characteristics becomes steeper and the red color (R) 
and the blue color (B) do not overlap each other. It is to 
be noted that the transmission factors of the optical fil- 
ters 52a, 52b, and 52c or of the optical filters 53a, 53b, 
and 53c are preferably set so as to provide appropriate 
signal levels in the respective image pickup regions in 
the same accumulation time. 

[0023] In the image processing system 20, a color im- 
age is formed based on the selective photoelectric con- 
version output obtained from the plurality of images by 
the plurality of image pickup regions of the solid-state 
image pickup element 120, respectively. Here, since the 
peak wavelength of the spectral luminous efficiency is 
555 nm, the signal processing is carried out using a G 
image signal including this wavelength, as the reference 
image signal. 

[0024] When the pixel pitch of the solid-state image 
pickup element is fixed, compared with a system adopt- 
ed generally by a digital color camera in which an RGB 
color filter is formed on the solid-state image pickup el- 
ement with, for example, 2x2 pixel block basis to pro- 
vide wavelength selectability to each pixel, and thus, an 
object image is separated into RGB images, the size of 
the object image is 1/V3 times, and thus, the focal length 
of the objective lens is about 1/V3 times, which is quite 
advantageous in thinning the camera. 
[0025] It is to be noted that, with regard to the spectral 
transmission factor characteristics of the optical filters 
52a, 52b, and 52c and of the optical filters 53a, 53b, and 
53c, as shown in Fig. 6, R and G overlap each other and 
G and B overlap each other, though R and B are almost 
separated from each other. 
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[0026] Therefore, even when the image circle 51b of 
the red light overlaps the image pickup region 120c for 
photoelectrical^ converting the blue light, and con- 
versely, even when the image circle 51c of the blue light 
overlaps the image pickup region 1 20b for photoelectri- 5 
cally converting the red light, these images do not be- 
come the output of the image pickup regions. However, 
at the portion where the image circle 51b of the red light 
overlaps the image pickup region 120a for photoelectri- 
cally converting the green light and at the portion where 10 
the image circle 51a of the green light overlaps the im- 
age pickup region 120b for photoelectrically converting 
the red light, a small amount of an image of a different 
wavelength which should be blocked is superimposed. 
More specifically, since the selectivity of the object im- *5 
age is given by the product of the spectral transmission 
factor characteristics of the optical filter 52a and the 
spectral transmission factor characteristics of the optical 
filter 53b, and by the product of the spectral transmission 
factor characteristics of the optical filter 52b and the 20 
spectral transmission factor characteristics of the optical 
filter 53a, the cross talk of the R image signal and the G 
image signal does not become zero although it is small. 
[0027] Therefore, the objective lens 1 00 is further pro- 
vided with characteristics to lower the transmission fac- 25 
tor of the wavelength range of the portion where R and 
G overlap each other. This can be carried out by using 
optical filtering technology implemented by a color purity 
correction filter. 

[0028] The color purity correction filter is an optical fil- 30 
ter formed of a base material made of a transparent syn- 
thetic resin or glass having a predetermined amount of 
rare-earth metal ions contained therein. 
[0029] As the rare-earth metal ions, one or more se- 
lected among neodymium ions, praseodymium ions, er- 35 
bium ions, holmium ions are used. However, it is pref- 
erable that at least neodymium Ions are used as indis- 
• pensable ions. It is to be noted that trivalence ions are 
generally used as these ions. The content of the metal 
ions is selected in a range of normally 0.01 to 40 mass *o 
parts, preferably 0.04 to 30 mass parts relative to 100 
mass parts of the base material of the objective lens 
100. 

[0030] As shown in Fig. 7, the color purity correction 
filter has characteristics to selectively absorb light in a 45 
predetermined wavelength range between the peak 
wavelengths of the color components of R, G, and B to 
decrease the amount of transmission thereof. This ac- 
tion almost eliminates cross talk due to the overlap of 
the image circle 51 b of the red light and the image pickup so 
region 120a for photoelectrically converting the green 
light and the overlap of the image circle 51 a of the green 
light and the image pickup region 120b for photoelectri- 
cally converting the red light. 

[0031] Further, the objective lens 100 is also provided 55 
with photochromic characteristics which is a phenome- 
non to be darkened by light and to be reversibly achro- 
matized when the irradiation of light is stopped. This is 



for, since the accumulation time control range of the sol- 
id-state image pickup element 120 is limited, suppress- 
ing the amount of light reaching the solid-state image 
pickup element when the field is extremely bright, there- 
by enlarging the recordable intensity range. 
[0032] As the photochromic glass, for example, pho- 
tochromic glass of phosphate glass made by Chance- 
Pilkington which has been put to practical use for spec- 
tacles (Product name: Reactolite Rapide) may be used. 
[0033] Fig. 8 is a graph showing the spectral trans- 
mission factor characteristics of a photochromic glass 
used as the objective lens 100. In Fig. 8, a solid line 
shows the characteristics after exposure to sunlight for 
20 minutes while a broken line shows the characteristics 
with no exposure to sunlight. When the camera is car- 
ried about with the user outdoors under a blue sky or 
the like, light beams which enter the objective lens 100 
from the diaphragm 110 makes the objective lens 100 
itself darkened to suppress the amount of light entering 
the solid-state image pickup element 120 to be about 
1/2. As a result, the accumulation time can be made 
twice as long, which raises the control limitation on the 
high intensity side. 

[0034] The screen size of each of the image pickup 
regions 120a, 120b, and 120c is 2.24 mm x 1.68 mm-* 
as described in the above, since the pixel pitch is 2.8 
urn and the number of the pixels are 800 x 600. The 
diagonal screen size of each of the image pickup re- 
gions is 2.80 mm. Generally, when the image pickup an- 
gle 6 of a small-sized camera is about 70° in the diago- 
nal direction, the camera is most convenient. When the 
image pickup angle is 70°, the focal length is 2.0 mm in 
this case, since it is determined by the diagonal size of 
the screen. 

[0035] When a person or the like is the subject of re- 
cording, considering that a person is about 170 cm tall 
and one person to three people are often recorded at a > 
time, a virtual distance D to the subject [m] can be de- 
fined as in the following equation (1) as a function of the 
image pickup angle 6[°]: 

D = 1.4/tan(6/2) (1) 

[0036] Substituting 70° for 9 of equation (1), D = 2.0 
m is obtained. Here, if the image pickup system 10 is 
formed so as to focus on the subject best when the dis- 
tance to the subject is 2 m, the letting out of the lens 
from the point at infinity is 0.002 mm. Taking into con- 
sideration the allowable circle of confusion to be de- 
scribed later, there causes practically no problem even 
if the image pickup system is a fixed focus image pickup 
optical system without a mechanism for letting the lens 
out. 

[0037] A focal length f of a planoconvex lens put in the 
air can be represented as follows: 
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f = {1/(1-n)}r (2) 

wherein n is the index of refraction and r is the radius of 
the spherical surface. Therefore, if the index of refrac- 5 
tion n of the objective lens 100 is 1.5, for example, r for 
obtaining the focal length of 2.0 mm is 1.0 mm. 
[0038] It is convenient to make equal the sizes of the 
red, green, and blue object images, since there is no 
need to make correction of the image magnification, and 10 
thus, the processing time is not elongated. Therefore, 
the lens portions 100a, 100b, and 100c are optimized 
for the peak wavelengths of 530 nm, 620 nm, and 450 
nm of the lights which go through the RGB optical filters, 
respectively to make equal the magnifications of the re- *5 
spective images. This can be paraxially materialized by 
making equal the distance from the positions of the prin- 
cipal points of the respective lens portions to the solid- 
state image pickup element. 

[0039] In case of glass with the index of refraction of 20 
the d line (587.6 nm) n d = 1.5 and the Abbe number v d 
= 60, the indexes of refraction at the wavelengths of 530 
nm, 620 nm, and 450 nm are about 1.503, 1.499, and 
1.509, respectively. If all the radii r of the spherical sur- 
faces of the lens portions 100a, 100b, and 100c equal 25 
-1 .0 mm, the focal lengths at the respective wavelengths 
are, by the equation (2), as follows: 

the lens portion 100a having the representative 
wavelength of 530 nm : 1 .988 mm, 30 
the lens portion 100b having the representative 
wavelength of 620 nm : 2.004 mm, and 
the lens portion 100c having the representative 
wavelength of 450 nm : 1.965 mm. 

35 

[0040] Suppose from the pixel pitch that the allowable 
circle of confusion is 3.0 u,m and further suppose that 
the f-number of the objective lens is F5.6, the depth of 
focus represented as the product of the two is 16.8 pm. 
It can be seen that the difference 0.039 mm between *o 
the focal length in case of 620 nm and the focal length 
in case of 450 nm already exceeds this. More specifi- 
cally, though the paraxial image magnifications are the 
same, depending on the color of the subject, the subject 
may be out of focus. Since the spectral transmission fac- 45 
tor of an object normally ranges over a wide wavelength 
range, it is quite rare that a sharp image in focus can be 
obtained. 

[0041] Accordingly, the radii r of the spherical surfac- 
es of the lens portions 100a, 100b, and 100c are opti- so 
mized with regard to the respective representative 
wavelengths. More specifically, here, no achromatism 
for removing the chromatic aberration over the whole 
visible region is not carried out and designs for the wave- 
lengths are applied to the respective lenses. First, the 55 
equation (2) is transformed to obtain the following equa- 
tion (3): 



r = 0-n)f (3) 

[0042] Inserting f = 2.0 and inserting in sequence n = 
1.503, n = 1.499, and n = 1.509 in the equation (3), the 
following respective radii are calculated: 

the lens portion 100a having the representative 

wavelength of 530 nm : r = -1 .006 mm, 

the lens portion 100b having the representative 

wavelength of 620 nm:r = -0.998 mm, and 

the lens portion 100c having the representative 

wavelength of 450 nm : r = -1 .018 mm. 

[0043] In order to make well-balanced the difference 
in the image magnifications at a high position of an im- 
age, the heights of the vertices of the lens portions 100a, 
100b, and 100c are slightly adjusted, and then, both the 
sharpness and the image magnification are ideal. Fur- 
ther, non-spherical surfaces are used for the respective 
lens portions to satisfactorily correct the curvature of the 
image plane. With regard to the distortion of the image, 
it can be corrected in the signal processing to be carried 
out later. 

[0044] In this way, when reference G image signals 
by the green object light including the wavelength of 555 
nm which has the highest luminosity factor and image 
signals by the red and blue object lights, respectively, 
are obtained, and different focal lengths are set with re- 
gard to one wavelength and a substantially equal focal 
length is set with regard to the representative wave- 
lengths in the respective spectral distribution in the im- 
aging system, then by compositing these image signals, 
a color image with satisfactory correction of the chro- 
matic aberration can be obtained. Since each of the im- 
aging systems is formed of one lens, there can be also 
attained technological advantage of thinning the imag- 
ing system. Further, though achromatism normally re- 
quires a combination of two lenses having different dis- 
persions, the achromatism is carried out here by only 
one lens, and thus, there can be also attained techno- 
logical advantage of lowering the cost. 
[0045] The objective lens 1 00 is required to have high 
contrast image dissection up to the spatial frequency 
range as high as the pixel pitch. The image pickup sys- 
tem 10 takes in three object images with regard to the 
three wavelength ranges, and thus, compared with an 
image pickup system provided with a mosaic optical fil- 
ter such as of the Baiyer arrangement having the same 
number of pixels, as described in the above, the focal 
length is about 1/>/3 times to attain the same image pick- 
up angle. Therefore, it is necessary to materialize high 
contrast image dissection of higher spatial frequency 
component. The optimization with regard to the respec- 
tive wavelengths of the lens portions described in the 
above is a technology for suppressing the chromatic ab- 
erration for the purpose of materializing this. 
[0046] Generally, there are two methods of improving 
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the aberration characteristics of an image pickup optical 
system to make spurious resolution less liable to occur, 
thereby alleviating problems: one is to increase the de- 
gree of freedom in designing by, for example, increasing 
the number of lenses forming the system, by making the 5 
lenses nonspherical, using anomalous dispersion glass, 
or by complexly using diffraction optical elements; and 
the other is to make narrower the imaging light beam. 
[0047] The former method, that is, increase of the de- 
gree of freedom in designing is, though the focal length 10 
is 1/V3 times, toward complexity of the structure of the 
objective optical system, which goes against thinning of 
the image pickup apparatus, and thus, is inappropriate. 
On the other hand, the latter method, that is, use of a 
narrower light beam conforms to a thin image pickup ap- * 5 
paratus. 

[0048] When an imaging light beam is made narrower, 
as shown by a solid line b in Fig. 10, the response func- 
tion referred to as OTF presents gradual monotone de- 
crease in the low frequency component, and after that, 20 
becomes slightly negative, and then, becomes slightly 
positive. On the other hand, in case a broad light beam 
without narrowing is used, as shown by a broken line a 
in Fig. 10, the response function presents steep de- 
crease in the low frequency component, and after that, 25 
becomes temporarily negative, and then, becomes pos- 
itive again. 

[0049] The state where the OTF is negative indicates 
occurrence of spurious resolution. This corresponds to 
an actual state where reversal of negative/positive oc- 30 
curs, that is, a portion which should be white is black 
while a portion which should be black is white. It makes 
it clear that a natural image can be obtained by making 
narrower the imaging light beam. 

[0050] However, when the imaging light beam is ex- 35 
tremely narrowed, the contrast in a high frequency 
range is decreased due to diffraction of light. In this 
state, an image is formed of a bright spot at the center 
and diffraction stripes surrounding the bright spot. 
These are caused by, as known well, relative increase *o 
in the intensity of the diffraction stripes due to peripheral 
waves generated on the periphery of the opening of the 
diaphragm. 

[0051] The diffraction stripes can be decreased by 
adding to the objective lens a filter which is transparent 45 
at the center and becomes darker toward the periphery. 
This method is referred to as apodization, which is de- 
scribed in detail on pages 1 72 to 1 74 of "Kogaku Gijutsu 
Handbook Zohoban (Handbook of Optical Technology, 
Enlarged Edition)" (1975, Asakura Shoten). 50 
[0052] Fig. 9 is a graph showing the transmission fac- 
tor distribution of transmission factor distribution type fil- 
ters provided on the light incidence plane 100e of the 
objective lens 1 00 so as to face the diaphragm openings 
110a, 110b, and 110c. The transmission factor distribu- 55 
tion type filters are denoted as 54a, 54b, and 54c in Fig. 
1. The positions where the transmission factor is the 
highest correspond to the centers of the diaphragm 



openings 110a, 110b, and 110c while the positions 
where the transmission factor is zero correspond to the 
periphery of the openings 110a, 11 0b, and 110c, respec- 
tively, of the diaphragm. In other words, the transmission 
factor is distributed so as to be the highest at the centers 
of the diaphragm openings and so as to monotonically 
decrease toward the periphery. 
[0053] Such a transmission factor distribution type fil- 
ter is formed by forming a thin film of Inconel, Chromel, 
chromium or the like by vapor deposition or sputtering 
on the light incidence side of the objective lens 100. The 
characteristics shown in Fig. 9 can be obtained by mak- 
ing the thin film the thinnest at the center and the thickest 
on the periphery. It is to be noted that, in forming such 
a thin film, the position of a mask used in the vapor dep- 
osition or sputtering is continuously controlled. 
[0054] Though the transmission factor distribution 
type filters 54a, 54b, and 54c are formed on the objective 
lens here, the structure may be that they are formed on 
a glass plate and are arranged on the light incidence 
side or the light projection side of the objective lens 100. 
[0055] Fig. 11 is a graph showing the intensity distri- 
bution in an image. In the figure, a broken line a shows 
a case where the transmission factor is constant over 
the whole diaphragm opening, while a solid line b shows 
a case where the transmission factor is decreased from 
the center of the diaphragm opening toward the periph- 
ery. Compared with the case of the characteristics 
shown by a, there is no bound on the periphery of the 
image in the characteristics shown by b, which clearly 
shows that the image formed is satisfactory. Here, This 
referects technological advantage of decreasing the dif- 
fraction stripes by decreasing peripheral light beams by 
means of apodization. 

[0056] Next, the relationship between the objective 
lens and the image pickup regions is described. Since 
the image pickup system has three lens portions, the - 
positions of the three object images relatively change ' 
according to the distance to the subject. As described 
in the above, each of the image pickup regions is sized 
to be 2.24 mm x 1 .68 mm, and the image pickup regions 
are arranged adjacent to each other with their long sides 
being in contact with each other. Therefore, the center 
to center distance of adjoining image pickup regions is 
1.68 mm. In the YC processing circuit 230 to be de- 
scribed later, signal processing is carried out on the as- 
sumption that the center of an object image is the center 
of the image pickup region. When an object image with 
the virtual object distance of 2 m is to be formed on the 
image pickup portion at the same intervals as that dis- 
tance, as shown in Fig. 12, the interval between the lens 
portions 100a and 100b and between the lens portions 
100b and 100c is set to be 1.6783 mm. In the figure, 
arrows 55a, 55b, and 55c denote imaging systems hav- 
ing positive power by the three lens portions 1 00a, 1 00b, 
and 100c of the objective lens 100, respectively, rectan- 
gles 56a, 56b, and 56c denote the ranges of the image 
pickup regions 120a, 120b, and 120c, respectively, and 
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L1 , L2, and L3 are optical axes of the imaging systems 
55a, 55b, and 55c, respectively. Since the light inci- 
dence surface of the objective lens 100 is a plane and 
the lens portions 100a, 100b, and 100c to be the light 
projection surface are spherical surfaces, perpendicu- 
lars from the respective centers of the spheres to the 
light incidence plane 100e are the optical axes. 
[0057] Here, as shown in Fig. 1 3, images of an object 
at infinity are formed at the same intervals as that of the 
. lens portions 100a, 100b, and 100c, and thus, the inter- 
val between the G object image and the R object image 
and the interval between the R object image and the B 
object image are 1.6783 mm, which is a little smaller 
than the center-to-center distance of the image pickup 
regions of 1.68 mm. The difference AY is 0.0017 mm, i. 
e. t 1.7 urn. With the G object image which has the high- 
est luminosity factor being as the reference object im- 
age, when the B object image moves, the difference AY 
is doubled and is 3.4 urn. Since it is often the case that 
a short-range object such as a person is positioned in 
the middle of a picture to be picked up and that a long- 
range object is positioned on the periphery of the pic- 
ture, and, since the aberration of the objective lens in- 
creases on the periphery of the picture to lower the qual- 
ity of an image, it can be understood that there is prac- 
tically no problem if the maximum image interval change 
is smaller than the pixel pitch multiplied by two. As de- 
scribed in the above, since the pixel pitch P of the solid- 
state image pickup element 120 is 2.8 urn, it follows that 
AY < 2 x P, and thus, the color misalignment in an image 
at-infinity to this extent is allowable. 
[0058] Further, the interval between the images varies 
also according to temperature change of the image pick- 
up system 10. Since the imaging magnification of the 
image pickup system 10 is extremely small, the image 
interval variation AZ can be expressed as the difference 
between the expansion of the objective lens and the ex- 
pansion of the solid-state image pickup element in the 
following equation (4): 

Z = 1.68 X (aL-aS) X AT (4) 

wherein aS is the coefficient of linear expansion of the 
solid-state image pickup element 120, <xL is the coeffi- 
cient of linear expansion of the objective lens 100, and 
AT is the temperature change. 

[0059] Here, when aS = 0.26 x 1 0' 5 , AT = 20[°C], and 
the objective lens 1 00 is formed of low-melting glass and 
ccL = 1 .2 x 10- 5 , AZ is calculated at 0.00032 [mm]. This 
is the variation in the interval between the G object im- 
age and the R object image and the variation in the in- 
terval between the R object image and the B object im- 
age. 

[0060] When the B object image is regarded as a var- 
iation with respect to the G object image which is the 
reference image signals, since the interval between the 
images is 1.68 x 2, the image interval variation is AZ x 
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2 = 0.00063 [mm]. When the temperature range where 
the camera is used is 0 to 40°C, the deviation AT from 
the reference temperature of 20°C is 20°C. Here, if AZ 
x 2 is smaller than 1/2 of the pixel pitch, there is prac- 
5 tically no problem. Generally, the coefficient of linear ex- 
pansion aS of the solid-state image pickup element 120 
is as small as about 0.26 x 1 0~ 5 , it is necessary to select 
the coefficient of linear expansion ccL of the objective 
lens 100 so as to satisfy the following expression (5): 

2 X A x (aL - 0.26 x 10' 5 ) x 20 < 1/2 P (5) 

wherein A is either the interval between the G image 
and the R image or the interval between the R image 
and the B image and P is the pixel pitch. 
[0061] Since the coefficient aL (= 1.2 x 10" 5 ) of linear 
expansion of the objective lens 100 used in the above 
satisfies the relationship expressed by the expression 
(5), it can be understood that the material of the lens 
100 is suitable for the camera. 
[0062] Further, apart from the fluctuation of the image 
interval due to the distance to the object and due to tem- 
perature change, a method referred to as pixel shift is 
used where, by shifting relatively the image pickup re- 
gions 120a, 120b, and 120c of the solid-state image 
pickup element by 1/2 pixel, the resolution is made high- 
er with a small number of pixels. The amount of the shift 
of 1/2 pixel is set with regard to the virtual object distance 
of 2 m. 

[0063] As shown in Fig. 15, the image pickup region 
120b for outputting the R image signals and the image 
pickup region 120c for outputting the B image signals 
are arranged so as to be shifted by 1/2 pixel both in the 
horizontal direction and in the vertical direction with re- 
spect to the image pickup region 120a for outputting the 
G image signals. 

[0064] This pixel shift may be materialized by making 
the lens portions 100b and 100c of the objective lens 
100 slightly offset with respect to the lens portion 100a, 
or may be materialized by making the image pickup re- 
gions 120b and 120c of the solid-state image pickup el- 
ement 1 20 slightly offset with respect to the image pick- 
up region 120a. 

[0065] In an optical filter arrangement such as the Bai- 
yer arrangement, since, for example, a pixel provided 
with a red optical filter or a pixel provided with a blue 
optical filter is arranged between pixels provided with a 
green optical filter, an optical low- pass filter for sup- 
pressing aliasing is necessary. However, in case of that 
images having different spectral distributions are taken 
in with regard to different image pickup regions, pixels 
provided with the respective optical filters can be dense- 
ly arranged, and, as a result, the influence of the aliasing 
is small, and thus, a high definition image can be ob- 
tained without the need for the optical low-pass filter. 
This makes it possible to miniaturize the image pickup 



EP 1 067 779 A2 



25 



30 



35 



40 



45 



50 



8 



15 



EP 1 067 779 A2 



16 



system and to substantially lower the cost. 
[0066] Next, the signal processing is described. 
[0067] As described in the above, an image pickup 
device having 1,800 pixels in its lengthwise direction 
and 800 pixels in its breadthwise direction, i.e., having 
1 ,440,000 pixels in total is effectively applied to the sol- 
id-state image pickup element 120. Optical filters of the 
three primary colors, that is, red (R), green (G), and blue 
(B), are arranged on its front surface in predetermined 
regions respectively. 

[0068] As shown in Fig. 14, image signals read from 
the solid-state image pickup element 120 are supplied 
to the A/D converter 500 of the image processing sys- 
tem 20. The A/D converter 500 is, for example, a signal 
converting circuit for converting the signals into digital 
signals of, for example, ten bits according to the ampli- 
tude of the signal of each pixel and for outputting the 
converted signals. Then, the image signal processing is 
carried out in digital processing. 
[0069] The image processing system 20 has a signal 
processing circuit for obtaining image signals of a de- 
sired format from R, G, and B digital signals, and con- 
verts for converting R, G, and B color signals into YC 
signals expressed by a luminance signal Y and chromi- 
nance signals (R-Y) and (B-Y) or the like. 
[0070] The RGB image processing circuit 21 0 is a sig- 
nal processing circuit for processing the pixel signals of 
the 1 800 x 800 pixels received from the solid-state im- 
age pickup element 120 through the A/D converter 500, 
and has a white balance circuit, a gamma control circuit, 
and an interpolation circuit for making the resolution 
higher by interpolation. 

[0071] The YC processing circuit 230 is a signal 
processing circuit for generating a luminance signal Y 
and chrominance signals R-Y and B-Y. The YC process- 
ing circuit 230 is formed of a high luminance signal gen- 
erating circuit for generating high luminance signal YH, 
a low luminance signal generating circuit for generating 
a low luminance signal YL, and a chrominance signal 
generating circuit for generating chrominance signals R- 
Y and B-Y. A luminance signal Y is formed by compos- 
iting a high luminance signal YH and a low luminance 
signal YL. 

[0072] The RGB image processing circuit 210 is now 
described in detail. 

[0073] RGB signals outputted through the A/D con- 
verter 500 with regard to the R, G, and B regions, re- 
spectively, are first subject to predetermined white bal- 
ance adjustment in the white balance circuit in the RGB 
image processing circuit 210, and then are subject to 
predetermined gamma control in the gamma control cir- 
cuit. 

[0074] The interpolation circuit in the RGB image 
processing circuit 21 0 generates by interpolation, image 
signals having the resolution which is four times as 
much as that of the 600 x 800 pixels, converts the image 
signals from the solid-state image pickup element 120 
into signals having high definition image quality, and 



supplies them to the high luminance signal generating 
circuit, the low luminance signal generating circuit, and 
the chrominance signal generating circuit at the subse- 
quent stage. 

5 [0075] Since the sizes of the R, G, and B object im- 
ages are already the same based on the setting of the 
objective lens 100, first, by a publicly known method, 
arithmetic processing for correcting optical aberration of 
the objective optical system is carried out with regard to 

10 the respective image signals. After that, the interpolation 
processing, the luminance signal processing, and the 
chrominance signal processing which are the same as 
those of an ordinary digital color camera are performed. 
The interpolation processing is as follows. 

15 [0076] First, a G image signal which is a reference im- 
age signal from the image pickup region 120a is subject 
to interpolation according to the following equations (6) 
to (9): 
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G2i2j = Gij (6) 

G2i(2j+1) = Gij-1/2 + GiQ+1)-1/2 (7) 

G(2i+1)2j = Gij-1/2 + G(i+1)j-1/2 (8) 

G(2i+1)(2j+1) = Gij-1/4 + GiG+1)-1/4+ 

G(i+1)j-1/4 + G(i+1)G+1)-1/4 (9) 



This is, 16 G elements are generated from each four G 
35 pixels, and thus the G image signals of 600 x 800 pixels 
from the image pickup region 120a are converted into 
1200 x 1600 pixels. 

[0077] Then, R image signals from the image pickup 1 
region 120b corresponding to the respective positions 
40 of the G image signals calculated using the above equa- 
tions (6) to (9) are subject to interpolation according to 
the following equations (10) to (13): 
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R2i2j = R(i-1)(j-1)-1/4 + R(i-1)j-1/4+ 

Ri(j-1)'1/4 + Rij-1/4 (10) 

R2i(2j+1) = R(i-1)j-1/2 + Rij-1/2 (11) 

R(2i+1)2j = Ri(j-1)-1/2 + Rij-1/2 (12) 

R(2i+1)(2j+1) = Rij (13) 
[0078] Since, as described in the above, the image 
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pickup region of the R object image and the image pick- 
up region of the B object image are arranged so as to 
be shifted by 1/2 pixel from the image pickup region of 
the G object image, the original output of the ij address 
is applied to the (2i+1 )(2j+1 ) address as indicated by the 
equation (13). 

[0079] Similarly to the R image signals, B image sig- 
nals from the image pickup region 120c corresponding 
to the respective positions of the G image signals cal- 
culated using the above equations (6) to (9) are subject 
to interpolation according to the following equations ( 1 4) 
to (17): 

B2i2j = B(i-1)(j-1)-1/4 + B(i-1)j-1/4+ 

Bi(j-1)<1/4 + Bij'1/4 (14) 



B2i(2j+1) = B(i-1)-j1/2 + Bij-1/2 (15) 



B(2i+1)2j = Bi(j-1)-1/2 + Bij-1/2 (16) 



B(2i+1)(2j+1) = Bij (17) 

[0080] By the above processing, the R, G, and B sig- 
nals of 600 x 800 pixels from the image pickup regions 
120a, 120b, and 120c are converted into R, G, and B 
signals of 1200 x 1600 pixels having high definition im- 
age quality, respectively. 

[0081] The high luminance signal generating circuit in 
the YC processing circuit 230 is a publicly known signal 
forming circuit for forming a high luminance signal YH 
from a color signal having the highest spatial frequency 
component among color component signals. The low lu- 
minance signal generating circuit is a publicly known 
signal forming circuit for forming luminance signal YL of 
low frequency from a signal containing all the color com- 
ponents of R, G, and B. The chrominance signal gener- 
ating circuit is a publicly known arithmetic circuit for cal- 
culating chrominance signals R-Y and B-Y from highly 
definition RGB signals. 

[0082] The recording and reproduction system 30 is 
a processing system for outputting image signals to the 
memory and outputting image signals to the liquid crys- 
tal monitor 4, and includes a recording processing circuit 
300 for writing and reading image signals in and from 
the memory and a reproduction processing circuit 310 
for reproducing image signals read from the memory as 
output to the monitor. More specifically, the recording 
processing circuit 300 includes a compressing and ex- 
panding circuit for compressing YC signals representing 
a still image or a dynamic image according to a prede- 
termined compression format and for expanding the 
compressed data when it is read from the memory. The 
compressing and expanding circuit includes a frame 



memory for signal processing and the like. YC signals 
from the image processing system 20 are stored on a 
frame basis, the signals are read on a plurality of block 
basis, and the signals are compression encoded. The 
5 compression encoding is carried out by, for example, 
two-dimensional orthogonal transformation, normaliza- 
tion, and encoding according to Huffman method on the 
block basis. 

[0083] The reproduction processing circuit 310 is a 

10 circuit for carrying out matrix conversion of the lumi- 
nance signal Y and the chrominance signals R-Y and 
B-Y into, for example, RGB signals. Signals converted 
by the reproduction processing circuit 31 0 are outputted 
to the liquid crystal monitor 4. Thus, a visible image is 

15 displayed and reproduced. 

[0084] On the other hand, the control system 40 in- 
cludes control circuits for controlling, in response to ex- 
ternal operation, the image pickup system 1 0, the image 
processing system 20, and the recording and reproduc- 

20 tion system 30, respectively. It detects depression of the 
RELEASE button 6 to control driving of the solid-state 
image pickup element 120, operation of the RGB image 
processing circuit 210, compression processing of the 
recording processing circuit 300, and the like. More spe- 

25 cifically, the control system 40 includes an operation de- 
tection circuit 410 for detecting the operation of the RE- 
LEASE button 6, a system control unit 400 for controlling 
the respective portions in response to the detection sig- 
nal and generating and outputting timing signals when 

30 an image is picked up and the like, and a drive circuit 
420 for the solid-state image pickup element 1 20, which, 
under control of the system control unit 400, generates 
drive signals for driving the solid-state image pickup el- 
ement 120. 

35 [0085] Next, operation of the image pickup apparatus 
according to the present embodiment is described with 
reference to Figs. 14 and 17A to 17C. First, when the 
main switch 5 is turned on, power source voltage is sup- 
plied to the respective portions and then they become 
40 operable. Then, it is determined whether an image sig- 
nal can be recorded or not in the memory. Here, the 
number of recordable pictures is displayed on the dis- 
play area 13 for the remaining number of pictures in a 
liquid crystal monitor 4 according to the remaining ca- 
45 pacity. An operator sees the display and, if a picture is 
recordable, directs the camera to the field and presses 
down the RELEASE button 6. 

[0086] When the RELEASE button 6 is halfway 
pressed down, the exposure time is calculated. When 
so all the preparatory processing for taking a picture com- 
pletes, the camera is in a recordable state, and the op- 
erator is informed of this through the display. When the 
RELEASE button 6 is fully pressed down, the operation 
detection circuit 4 1 0 transmits the detection signal to the 
55 system control unit 400. Here, the exposure time calcu- 
lated in advance is counted. When the predetermined 
exposure time elapses, a timing signal is supplied to the 
drive circuit 420. Then, the drive circuit 420 generates 
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horizontal and vertical drive signals to sequentially read 
each of the exposed 1600 x 800 pixels in the horizontal 
direction and in the vertical direction. 
[0087] The read signals of respective pixels are con- 
verted by the A/D converter 500 into digital signals of a 5 
predetermined number of bits to be sequentially sup- 
plied to the RGB image processing circuit 210 of the im- 
age processing system 20. The RGB image processing 
circuit 21 0 carries out interpolation to quadruple the pix- 
el resolution after white balance adjustment and gamma *o 
control, and supplies the signals to the YC processing 
circuit 230. 

[0088] In the YC processing circuit 230, the high lu- 
minance signal generating circuit generates high lumi- 
nance signal YH with regard to the respective RGB pix- *5 
els, and similarly, the low luminance signal generating 
circuit generates low luminance signal YL. The calculat- 
ed high luminance signal YH is outputted to an adder 
through a low-pass filter. In the same way, the low lumi- 
nance signal YL is, after subtraction of the high lumi- 20 
nance signal YH, outputted to the adder through the low 
pass filter. By this, the difference YL-YH between a high 
luminance signal and a low luminance signal is added 
to obtain an luminance signal Y. Similarly, the chromi- 
nance signal generating circuit generates and outputs 25 
chrominance signals R-Y and B-Y. Components of the 
outputted chrominance signals R-Y and B-Y, which 
come through the low-pass filter, are supplied to the re- 
cording processing circuit 300. 

[0089] Next, the recording processing circuit 300 re- so 
ceives the YC signals, compress the respective lumi- 
nance signal Y and chrominance signals R-Y and B-Y 
with a predetermined still image compression method, 
and records them sequentially in the memory. 
[0090] When a still image or a dynamic image is re- 35 
produced from image signals stored in the memory, the 
PLAY button 9 is pressed down. Then, the operation de- 
tection circuit 410 detects this operation to supply a de- 
tection signal to the system control unit 400, which 
makes the recording processing circuit 300 driven. The *o 
driven recording processing circuit 300 reads the re- 
corded content from the memory to display images on 
the liquid crystal monitor 4. The operator selects the de- 
sired image by pressing down a SELECT button or the 
like. 45 
[0091] In a second embodiment of the present inven- 
tion, an image pickup system which is different from that 
of the first embodiment in the positional relationship in 
the objective lens and in the image pickup regions. Fig. 
18 is a detailed view of an image pickup system 890. 50 
First, the diaphragm 810 has four circular openings 
81 0a, 81 0b, 81 0c, and 81 Od as shown in Fig. 21 . Object 
light which enters a light incidence plane 800e of an ob- 
jective lens 800 from the openings are projected from 
four lens portions 800a, 800b, 800c, and 800d of the 55 
objective lens 800 to form four object images on the im- 
age pickup plane of a solid-state image pickup element 
820. The diaphragm 810, the light incidence plane 800e, 



and the image pickup plane of the solid-state image 
pickup element 820 are in parallel with one another. 
Though the light incidence plane 800e of the objective 
lens 800 is a plane here, the light incidence plane 800e 
may be formed as four spherical surfaces or four rota- 
tion-symmetry rotation-symmetry nonspherical surfac- 
es. 

[0092] The four lens portions 800a, 800b, 800c, and 
800d have circular spherical surface portions as shown 
in Fig. 20 showing the objective lens 800 viewed from 
the side where light is projected. An infrared radiation 
cutting filter having a low transmission factor with regard 
to the wavelength range of 670 nm and more is formed 
on the spherical surface portions while a light-shielding 
layer is formed on a plane portion 800f shown with 
hatching. More specifically, the objective optical system 
is formed with the objective lens 800 and the diaphragm 
810, while the four lens portions 800a, 800b, 800c, and 
800d are the imaging system. 

[0093] Fig. 22 is a front view of the solid-state image 
pickup element 820. The solid-state image pickup ele- 
ment 820 is provided with four image pickup regions 
820a, 820b, 820c, and 820d corresponding to the 
formed four object images. Each of the image pickup 
regions 820a, 820b, 820c, and 820d is sized to be 2.24 — 
mm x 1 .68 mm with the arrangement of 800 x 600 pixels 
having the vertical and horizontal pitch of 2.8 urn. The 
size of the whole image pickup regions is 4.78 mm x 
3.36 mm, including an image pickup region separator of 
0.3 mm in the middle. The diagonal size of each of the 
image pickup regions is 2.80 mm. In this figure, image 
circles 851a, 851b, 851c, and 851d within which an ob- 
ject image is formed are circular the sizes of which are 
determined based on the openings of the diaphragm 
and the size of the spherical surface portions on the pro- 
jection side of the objective lens 800. The image circles 
851a, 851b, 851c, and 851d overlap one another. 
[0094] In Fig. 18, hatched portions 852a and 852b 
sandwiched between the diaphragm 810 and the objec- 
tive lens 800 are optical filters formed on the light inci- 
dence plane 800e of the objective lens 800. As shown 
in Fig. 19 showing the objective lens 800 viewed from 
the light incidence side, the optical filters 852a, 852b, 
852c, and 852d are formed in regions which completely 
include the diaphragm openings 810a, 810b, 810c, and 
81 Od, respectively. 

[0095] The optical filters 852a and 852d have spectral 
transmission factor characteristics shown as G in Fig. 6 
which mainly transmit green, the optical filter 852b has 
spectral transmission factor characteristics shown as R 
which mainly transmit red, and the optical filter 852c has 
spectral transmission factor characteristics shown as B 
which mainly transmit blue, and thus, these are primary 
color filters. As the product with the characteristics of 
the infrared radiation cutting filter formed on the lens 
portions 800a, 800b, 800c, and 800d, the object images 
formed in the image circles 851a and 851 d are of green 
light component, the object image formed in the image 
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circle 851b is of red light component, and the object im- 
age formed in the image circle 851c is of blue light com- 
ponent. 

[0096] On the other hand, optical filters 853a, 853b, 
853c, and 853d are formed on the four image pickup 
regions 820a, 820b, 820c, and 820d, respectively, of the 
solid-state image pickup element 820. Their spectral 
transmission factor characteristics are equivalent to the 
ones shown in Fig. 6. More specifically, the image pick- 
up regions 820a and 820d are sensitive to green light 
(G), the image pickup region 820b is sensitive to red light 
(R), and the image pickup region 820c is sensitive to 
blue light (B). 

[0097] Since the spectral distributions of the received 
light of the respective image pickup regions are given 
as the product of the spectral transmission factors of the 
pupils and the spectral transmission factors of the image 
pickup regions, respectively, the combination of the pu- 
pil and the image pickup region is selected according to 
the wavelength range. More specifically, object light 
which goes through the diaphragm opening 810a is 
mainly photoelectrically converted in the image pickup 
region 820a, object light which goes through the dia- 
phragm opening 810b is mainly photoelectrically con- 
verted in the image pickup region 820b, object light 
which goes through the diaphragm opening 810c is 
mainly photoelectrically converted in the image pickup 
region 820c, and object light which goes through the di- 
aphragm opening 81 Od is mainly photoelectrically con- 
verted in the image pickup region 820d. In other words, 
the image pickup regions 820a and 820d output G im- 
ages, the image pickup region 820b outputs an R image, 
and the image pickup region 820c outputs a B image, 
respectively. In this way, by using multiple optical filters 
for the purpose of color separation on the pupils of the 
image pickup optical system and on the image pickup 
element, the color purity can be enhanced. 
[0098] It is to be noted that the separator between the 
image pickup regions is provided for the purpose of pre- 
venting overlap of the image circles in the image pickup 
regions 820a and 820d, since both of them obtain G im- 
age signals. 

[0099] In the image processing system 20, a color im- 
age is formed based on the photoelectric conversion 
output of one of the plurality of images obtained in the 
plurality of image pickup regions of the solid-state image 
pickup element 820. Here, since the peak wavelength 
of the spectral luminous efficiency is 555 nm, the signal 
processing is carried out with a G image signal including 
this wavelength being as the reference image signal. 
Since a G object image is formed both in the image pick- 
up region 820a and in the image pickup region 820d, 
the number of pixels of the G object image is twice as 
much as that of the R object image or the B object image, 
and thus, a particularly high definition image can be ob- 
tained in the wavelength range having the highest lumi- 
nosity factor. 

[0100] It is to be noted that, here, apart from the fluc- 



tuation of the image interval due to the distance to the 
object and temperature change, which is described in 
the following, the method referred to as pixel shift is used 
where, by shifting relatively the image pickup regions 
5 820a and 820d of the solid-state image pickup element 
by 1/2 pixel, the resolution is made higher with a small 
number of pixels. The amount of the shift of 1/2 pixel is 
set with regard to the virtual object distance 2 m. 
[0101] This pixel shift may be materialized by making 
the lens portion 800d of the objective lens 800 slightly 
offset with respect to the lens portion 800a, or may be 
materialized by making the image pickup region 820d 
of the solid-state image pickup element 820 slightly off- 
set with respect to the image pickup region 820a. 
[0102] Since the image pickup system has four lens 
portions, the positions of the four object images relative- 
ly change according to the distance to the object. As de- 
scribed in the above, each of the image pickup regions 
is sized to be 2.24 mm x 1 .68 mm, and each two of them 
are arranged adjacent to each other with their longer 
sides being in contact with each other. The two pairs are 
in parallel with each other with the separator of 0.3 mm 
therebetween. Therefore, the center-to-center distance 
of adjoining image pickup regions is 1 .68 mm in the ver- 
tical direction and 2.54 mm in the horizontal direction. 
The distance between image pickup regions is longest 
with regard to the image pickup regions 820a and 820d, 
and with regard to the image pickup regions 820b and 
820c, which are positioned obliquely with respect to 
each other. This longest distance is 3.045 mm in both 
cases. 

[0103] In the YC processing circuit 230 to be de- 
scribed later, signal processing is carried out on the as- 
sumption that the center of an object image is the center 
of the image pickup region. With regard to the image 
pickup regions 820a and 820d, in case of that object im- 
ages with the virtual object distance 2 m are to be formed 
on the image pickup portion so as to keep the interval 
of 3.045 mm which is the same as the interval between 
the image pickup regions, as shown in Fig. 23, the in- 
terval between the lens portions 800a and 800d of the 
objective lens 800 is set to be 3.0420 mm. In the figure, 
arrows 855a and 855d denote imaging systems having 
positive power by the lens portions 800a and 800d of 
the objective lens 800, respectively, rectangles 856a 
and 856d denote the ranges of the image pickup regions 
820a and 820d, respectively, and L801 and L802 are 
optical axes of the imaging systems 855a and 855d, re- 
spectively. Since the light incidence surface 800e of the 
objective lens 800 is a plane and the lens portions 800a 
and 800d which are the light projection surface are 
spherical surfaces, perpendiculars from the respective 
centers of the spheres to the light incidence plane are 
the optical axes. 

[0104] Here, as shown in Fig. 24, images of an object 
at infinity are formed at the same intervals as that of the 
lens portions 800a and 800d, and thus, the interval be- 
tween the G object images is 3.0420 mm, which is a little 
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smaller than the center-to-center distance of the image 
pickup regions of 3.045 mm. The difference AY is 0.0030 
mm, i.e., 3.0 urn. Such fluctuation in the interval be- 
tween the G object images appears as decrease in the 
resolution. 5 
[0105] With regard to the image pickup regions 820b 
and 820c, similarly, AY is 3.0 u.m and thus, color mis- 
alignment by this amount is caused with regard to an 
image at infinity. Therefore, two problems have to be 
considered: fluctuation in the interval between the G ob- 
ject images, and misalignment of the R object image and 
the B object image. 

[0106] However, since the spectral luminous efficien- 
cy is high with regard to a G object image and thus the 
G object image can be recognizable even with slight de- '5 
crease in its resolution, first, the fluctuation in the inter- 
val between the G object images is-considered. If such 
the interval is allowable, it can be understood that the 
misalignment of the R object image and the B object im- 
age causes no problem. 20 
[0107] Since, generally, it is often the case that a 
short-range object such as a person is positioned in the 
middle of a picture to be picked up and that a long-range 
object is positioned on the periphery of the picture, and, 
since the aberration of the objective lens increased on 25 
the periphery of the picture to lower the quality of an 
image, there is practically no problem, if the maximum 
image interval change between the G object images is 
smaller than the pixel pitch multiplied by two. Since the 
pixel pitch P of the solid-state image pickup element 820 30 
is 2.8 urn, it follows that AY < 2 x P, and thus, the color 
misalignment in an image at infinity to this extent is al- 
lowable. 

[0108] It follows that the misalignment of the R object 
image and the B object image is also allowable, and 35 
thus, it can be understood that this image pickup system 
has practically sufficient performance. 
[01 09] The individual components shown in schemat- 
ic or block form in the drawings are all well-known in the 
camera arts and their specific construction and opera- <o 
tion are not critical to the operation or best mode for car- 
rying out the invention. 

[0110] While the present invention has been de- 
scribed with respect to what is presently considered to 
be the preferred embodiments, it is to be understood 45 
that the invention is not limited to the disclosed embod- 
iments. To the contrary, the invention is intended to cov- 
er various modifications and equivalent arrangements 
included within the spirit and scope of the appended 
claims. The scope of the following claims is to be ac- so 
corded the broadest interpretation so as to encompass 
all such modifications and equivalent structures and 
functions. 

[0111] For example, in the described embodiments, 
optical filters used for the color separation are primary 55 
color filters of red, green, and blue. However, the 
present invention is also applicable to a case where 
these filters are replaced by complementary color filters 



of cyan, magenta, and yellow. 
[0112] Further, in the above embodiments, an object 
image having the highest luminosity factor (green) is 
used as a reference to describe change in the intervals 
between the object image having the highest luminosity 
factor and object images having other luminosity factors 
(other colors), which caused by change in the distance 
to the object. However, the present invention is also ap- 
plicable to a case where an object image having the 
highest luminosity factor (green) is not used as the ref- 
erence. 

[0113] It is to be noted that, in the present invention, 
the respective embodiments in the above or technical 
elements thereof may be freely combined depending on 
the situation. 

[0114] Also, in the present invention, the whole or part 
of the appended claims or of the structure of an embod- 
iment may form one apparatus, may be combined with 
another apparatus, or may be an element of an appara- 
tus. 



Claims 

1. An image pickup apparatus, characterized by com- 
prising: 

a plurality of openings (110a to 110c, 810a to 
81 Od) for taking in external light at different po- 
sitions; 

filter means (52a to 52c, 53a to 53c, 852a to 
852d, 853a to 853d) for extracting from external 
light taken in from said plurality of openings, 
predetermined wavelength components of ob- 
ject light correspondingly to said plurality of 
openings, respectively; and 
a plurality of image pickup portions (120a to 
120c, 820a to 820d) for receiving said prede- 
termined wavelength components extracted by 
said filter means correspondingly to said plural- 
ity of openings, respectively. 

2. An image pickup apparatus according to claim 1 , 
characterized by further comprising: 

a plurality of optical systems (100a to 100c, 
800a to 800d) for guiding said predetermined wave- 
length components extracted by said filter means 
to the corresponding image pickup portions, re- 
spectively. 

3. An image pickup apparatus according to claim 2, 
characterized in that each of said plurality of optical 
systems comprises a single lens (100a to 100c, 
800a to 800d). 

4. An image pickup apparatus according to claim 3, 
characterized in that a plurality of single lenses of 
said plurality of optical systems are integrally 
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formed of a glass material or a resin material. 

5. An image pickup apparatus according to claim 4 fur- 
ther comprising: 

a light-shielding layer (100d, 800f) provided 
between said plurality of single lenses which are in- 
tegrally formed. 

6. An image pickup apparatus according to claim 2, 
characterized in that each of said plurality of optical 
systems comprises a single lens (100a to 100c, 
800a to 800d) provided with an infrared radiation 
cutting filter. 

7. An image pickup apparatus according to claim 2, 
characterized in that each of said plurality of optical 
systems comprises photochromic glass. 

8. An image pickup apparatus according to claim 2, 
characterized in that each of said plurality of optical 
systems comprises a color purity correction filter 
(100a to 100c, 800a to 800d). 

9. An image pickup apparatus according to claim 2, 
characterized in that each of said plurality of optical 
systems comprises a filter (54a to 54c) whose trans- 
mission factor of which becomes smaller as the dis- 
tance from an optical axis thereof becomes longer. 

10. An image pickup apparatus according to claim 2, 
characterized in that, when a virtual object distance 
D to [m] is defined as a function of an image pickup 
angle G[°] of said plurality of optical systems to be 
D = 1 A I tan (6/2), an interval between optical axes 
of said plurality of optical systems are set such that 
change in an interval between an object image re- 
ceived by a first image pickup portion which is one 
of said plurality of image pickup portions and an ob- 
ject image received by a second image pickup por- 
tion which is another one of said plurality of image 
pickup portions between when an object exists at 
said virtual distance to the object and when the ob- 
ject exists at infinity is smaller than a pixel pitch of 
said plurality of image pickup portions multiplied by 
two. 

11. An image pickup apparatus according to claim 10, 
characterized in that the predetermined wavelength 
component received by said first image pickup por- 
tion is green and the predetermined wavelength 
component received by said second image pickup 
portion is at least one of red and blue. 

12. An image pickup apparatus according to claim 10, 
characterized in that the predetermined wavelength 
component received by said first image pickup por- 
tion is a wavelength component having the highest 
luminosity factor and the predetermined wave- 



length component received by said second image 
pickup portion is a wavelength component having a 
luminosity factor different from said highest lumi- 
nosity factor. 

5 

13. An image pickup apparatus according to claim 10, 
characterized in that the predetermined wavelength 
components received by said first and second im- 
age pickup portions are the same wavelength com- 

10 ponent. 

14. An image pickup apparatus according to claim 10, 
characterized in that the predetermined wavelength 
components received by said first and second im- 

15 age pickup portions are green. 

15. An image pickup apparatus according to claim 10, 
characterized in that the predetermined wavelength 
components received by said first and second im- 

20 age pickup portions are the wavelength component 
having the highest luminosity factor. 

16. An image pickup apparatus according to claim 1, 
characterized in that said plurality of image pickup 

25 portions are integrally formed. 

17. An image pickup apparatus according to claim 1, 
characterized in that said plurality of image pickup 
portions are formed in a plane shape. 

30 

18. An image pickup apparatus according to claim 1, 
characterized in that said filter comprises an optical 
filter for transmitting red light, an optical filter for 
transmitting green light, and an optical filter for 

35 transmitting blue light correspondingly to said plu- 
rality of openings, respectively. 

19. An image pickup apparatus according to claim 18, 
characterized in that said filter comprises a plurality 
of optical filters provided such that a same color light 
transmits each one of said plurality of openings and 
each one of said plurality of image pickup portions 
corresponding to said each one of said plurality of 
openings. 

45 

20. An image pickup apparatus according to claim 1 8, 
characterized in that said plurality of openings are 
arranged so as to arrange two in a vertical direction 
x two in a horizontal direction, and said optical filter 

50 for transmitting green light is provided correspond- 
ingly to two of said plurality of openings. 

21. An image pickup apparatus according to claim 1, 
characterized in that said filter comprises an optical 

55 filter for transmitting cyan light, an optical filter for 
transmitting magenta light, and an optical filter for 
transmitting yellow light correspondingly to said plu- 
rality of openings, respectively. 



35 



14 



EP 1 067 779 A2 




15 



EP 1 067 779 A2 




16 



EP 1 067 779 A2 



FIG. 6 i 



100 



1 50 

CO 

< 



400 500 600 

WAVELENGTH ( nm ) 



700 



FIG. 7 



FIG. 8 



> 

CO 
CO 

CO 

< 
GC 



100 
90 
80 
70 
60 
50 
40 
30 
20 
10 
0 



100 



1 50 

CO 

z 
< 



400 500 600 

WAVELENGTH ( nm ) 



700 



AFTER EXPOSURE 
TO LIGHT 
NO EXPOSURE 
TO LIGHT 



400 450 500 550 600 650 700 750 800 
WAVELENGTH ( nm ) 



17 



EP 1 067 779 A2 



FIG. 9 



FIG. 10 



FIG. 11 



> 
CO 

CO 
< 

cc 



DISTANCE FROM CENTER 




SPATIAL FREQUENCY 



g 




DISTANCE FROM CENTER OF IMAGE 



18 



EP 1 067 779 A2 




FIG. 13 



L1 

1.6783 


55a_, 
55b 


56a — 
56b—- 




S 

L2 1.6783 










55c ^ 






56c— 
2.000 





1.680 



19 



EP 1 067 779 A2 



SO 



r 



o 

CO 



< 



o 



^00 
Qcoi— 

oo=> 



li 



1 



O00|_- 




20 



EP 1 067 779 A2 



FIG. 15 



1/2 



•R, B 



1/2 



FIG. 16 



r 



{ 



V 



Goo 


G01 




G10 


Gii 


\ 






Goo 


G01 


G02 


G03 


G10 


Gn 


G12 


G13 


G20 


G21 


G22 


G23 


G30 


G31 


G32 


G33 



21 



EP 1 067 779 A2 




22 



EP 1 067 779 A2 



FIG. 18 



810 



852a- 



1 



852b-a 



(1 



800e 

v 



820 



-800a 



853a- 



-800b 



X 



853b —I 



x 
x 



890 



FIG. 19 




23 



EP 1 067 779 A2 



FIG. 20 




FIG. 21 

^810 
810c 

81 Od 




EP 1 067 779 A2 



FIG. 22 




25 



EP 1 067 779 A2 



FIG. 23 




3.045 



2.000 



2000.000 



FIG. 24 



L801 


855a ( 
3.0420 


856a—' 




L8 ° 2 855d^ 






856d — 
2.000 





3.045 



26 



